1. Field of the Invention
The present invention is related to the automated design of reliability testing. Specifically, the invention is a specification-based test automation technique that captures implicit domain specific properties in system test models automatically by exploiting type-specific axioms of syntactical relationships between morphemes of domain specific system description languages.
2. Description of the Prior Art
Reliability test automation is an effective means by which aggressive product deadlines may be met and by which a quality final computing application may be delivered. However, the automated testing methods that are heretofore available may fail to scrutinize all of the requirements of a specific problem domain associated with the system under development. This is mainly due to the fact that domain specific requirements are not usually specified explicitly. To illustrate, consider the functional specification fragment, “The system must power down if backup power is unavailable.” A system modeler may translate the fragment into a description language specification fragment “if (backup!=available) then power:=off”. A test case generator may then provide a test script to test the system for the specified requirement, but prior automatic test suites would not automatically test an implicit “else” case, i.e., “if (backup==available) then . . . ”. Essential, the lack of the ability to automatically test for the implicit content of a specification leaves a gap in a complete reliability test of a system.
As another example, consider any database application throwing queries to a static database. It is unlikely that there will be an explicit requirement for the application to handle database related errors such as throwing queries for a non-existent field of the database. This null field requirement is nevertheless important to insure proper operation of the application and, hence, such functionality should be tested.
For the past several decades, computing applications in particular have become increasingly sophisticated and feature-rich. As more complex designs are being attempted, the character of software development has changed. As the modest designs of yesteryear give way to more elaborate systems, the greater chance of error exists in implementing the design. Adding to the possibility for error is the fact that the present users of software are not primarily programmers as in times gone by, but are persons of all ages and computer skill levels. Thus, it becomes critical that software be tested not only for functionality according to the design specification, but also for implied functionality, such as handling erroneous user input.
Recently, the application of the extended finite state machine (EFSM) to functional test generation has increased the thoroughness of automated procedures. The EFSM extends the simple finite state machine by adding data variables. The data variables allow transitions from one state to another based on the value associated therewith. The state which follows when a specific input is processed is not just determined by the existing state and input, as is the case with the simple state finite machine. A state transition may also occur in an EFSM in accordance with information passed in signals or through parameters of a function call. Thus, the extended finite state machine is particularly useful in modeling not only certain systems, but also various types of software applications.
In the Journal paper “Automatic Functional Test Generation Using the Extended Finite State Machine Model”, by K. Cheng and A. Krishnakumar (30th Proceeding of the ACM/IEEE Design Automation Conference, June 1993), the authors present a method of generating functional vectors for sequential circuits from a high-level description of the circuit in VHDL or C programming language. The method guarantees, according to the authors, that the generated set of vectors cover every statement in the high-level description at least once. Whereas, such generalized syntheses of an EFSM from a high-level specification have produced favorable results, such does not insure that a complete set of properties peculiar to the specific problem domain are included in the model. Thus, some features of the system may be excluded from testing. This is especially true when certain functionality of the system that was not explicitly set forth in the formal specification.
The shortcomings of automated test design discussed above are prevalent in the prior art. In light of these shortcomings and to further advance the technology in the field of automated reliability test design, there exists a need for a method for automatically generating test cases for behaviors of a system operating within a problem domain which are both explicitly set forth in the formal specification and implicit to the formal specification.